Computing devices continue to become more ubiquitous to daily life. They take the form of computer desktops, laptop computers, tablet computers, hybrid computers (2-in-1s), e-book readers, mobile phones, smartphones, wearable computers (including smartwatches, smart glasses/headsets), global positioning system (GPS) units, enterprise digital assistants (EDAs), personal digital assistants (PDAs), game consoles, and the like. Further, computing devices are being incorporated into vehicles and equipment, such as cars, trucks, farm equipment, manufacturing equipment, building environment control (e.g., lighting, HVAC), and home and commercial appliances.
Computing devices generally consist of at least one processing element, such as a central processing unit (CPU), some form of memory, and input and output devices. The variety of computing devices and their subsequent uses necessitate a variety of interfaces and input devices. One such input device is a touch sensitive surface such as a touch screen or touch pad wherein user input is received through contact between the user's finger or an instrument such as a pen or stylus and the touch sensitive surface. For the input of content, such as text, such devices provide for input on screen through handwriting which involves users entering ‘strokes’ forming characters and the like, or through typing which involves users ‘striking’ keys of a displayed or graphical (virtual) keyboard, or through “stroke-like” typing which involves users interacting with the keys of the virtual keyboard with continuous strokes, such as provided by SWYPE®, SWIFTKEY® and TOUCHPAL® keyboards, for example.
Text content of typing input via a virtual (soft) keyboard, for example, is conventionally recognized using a keyboard decoder of the device. Text and non-text content of handwriting input via user interaction with the touch sensitive surface, for example, is conventionally recognized using a handwriting recognizer of the device.
Some devices also have imaging devices, such as in-built cameras, that can be used to image or scan content from non-digital, e.g., printed, or other digital forms, for capture. This captured content may be displayed on the capturing device, such as a smartphone, or communicated with another device, such as a scanner communicating with a connected personal computer (PC), for example.
Typed or fontified text content of imaged input via the camera of the device or scanner, for example, is conventionally recognized using optical character recognition (OCR) so that the imaged text can be converted into editable and searchable digital content. OCR typically involves image processing techniques for resolving text characters from the image. These techniques typically involve analysis of the pixels of the digitized image in ‘monochromatic’, grayscale or color domains to extract pixels which can be considered to represent the characters from pixels that can be considered to not represent the characters. This process is typically termed “thinning” or “skeletonization” and extracts the ‘skeleton’ of the imaged characters so as to provide a compact and non-redundant representation of the characters.
A number of conventional skeletonization methods have been developed. The simplest of these methods involve (morphological) thinning of foreground pixels in the image in iterative, single-pass or parallel processing fashion. This simple method is generally performed by removing foreground pixels while thinning the objects. Gradient-based methods involve computing gradients which are highest at boundary pixels of objects and lowest at the skeleton pixels. These gradients can be calculated using distance transforms on the binary or grayscale image intensities. Contour-based methods, such as medial axis transform, techniques with Delaunay triangulation and Voronoi diagrams, use the perimeter of the object to deduce the skeleton. Principal component analysis (PCA) methods have also been explored in which the skeleton is approximated as a set of parametric curves, like Bezier or B-splines, and fitted to the image data.
These conventional OCR skeletonization methods are variously sensitive to noise in the captured or scanned images which results in inaccurate character recognition. This is because poor imaging of the characters introduces spurious artefacts in the image which leads to skeletons being deduced with branches, segments, etc., which are not part of the intended character.
In comparison to typed or printed content, handwritten text content of imaged input, so-called off-line handwriting recognition, is conventionally recognized using intelligent character recognition (ICR) or intelligent word recognition (IWR). ICR systems recognize content from images of characters handwritten in a structured manner. This structure includes characters which are isolated, e.g., non-cursive, and substantially evenly-spaced. ICR has been successful in enterprise form applications in which users handwrite information in fields or boxes designated for individual characters, such as letters, numbers and symbols. IWR systems on the other hand recognize content from images of characters handwritten in an unstructured or free-form manner, e.g., cursive. This is achieved by performing word-based recognition, rather than character-based recognition as in ICR.
Like OCR, ICR and IWR conventionally involve image processing techniques for resolving text characters from the image, including skeletonization. Also like OCR, the accuracy of the recognition using either ICR or IWR depends on the quality of the imaged characters. This is exacerbated for handwriting however as handwriting includes a wide range of natural variation, many differing styles and may not adhere to expected standards. As such, the technique of thinning pixels in images of handwritten content may introduce unwanted trajectories in the handwriting strokes, such as branches, segments, tails, “neckings”.
A number of techniques have been investigated for dealing with such issues in handwriting image recognition. For example, U.S. Pat. No. 4,034,344 describes applying compound operators in cycles for thinning handwritten characters, U.S. Pat. No. 5,146,996 describes a process which detects pre-defined geometrical features in the thinned handwritten characters, U.S. Pat. No. 5,224,179 describes thinning using a template or mask, U.S. Pat. No. 5,515,455 describes thinning cursively handwritten words by using temporal information, and U.S. Pat. No. 6,377,710 describes using a contour based method for handwriting.
Other techniques have been developed for recognizing imaged handwriting apart from skeletonization. For example, U.S. Pat. No. 5,113,453 describes a process which attempts to find primitive shapes in imaged handwritten text, U.S. Pat. No. 6,154,565 describes a process using code division multiple access (CDMA) techniques to differentiate handwriting from background pixels, and US Patent Application Publication No. 2007/0065003 describes using classification techniques to distinguish handwritten and printed text.